[Test] Add realistic benchmark input profiles - #1144

Summary

Add opt-in, reproducible benchmark input profiles that better match the distributions produced by FLA layers. The op runner now supports seeded realistic inputs for GLA, GDN, KDA, Comba, and delta-rule workloads, including normalized or learned decay gates, SiLU activations, and Comba's correlated p input. The training-throughput benchmark also supports exact-budget log-normal packed sequence lengths.

The existing synthetic op profile and random packed-length mode remain the defaults. This changes benchmark input generation only; no kernel, layer, model, or public operator implementation changes.

Addresses part of #350.

Test plan

• Dependent-test discovery: python scripts/find_dependent_tests.py benchmarks/benchmark_training_throughput.py benchmarks/distributions.py benchmarks/ops/registry.py benchmarks/ops/run.py tests/test_benchmark_distributions.py
  • Selected tests/test_benchmark_distributions.py.
• New CPU tests: python -m pytest -q tests/test_benchmark_distributions.py
  • 9 passed.
• Representative frozen GPU correctness cases for chunk_gla, chunk_gated_delta_rule, chunk_kda, and chunk_comba.
  • 4 passed on an NVIDIA GeForce RTX 5080 with CUDA 13.0, PyTorch 2.10.0+cu130, and Triton 3.6.0.
• pre-commit run --files benchmarks/benchmark_training_throughput.py benchmarks/distributions.py benchmarks/ops/registry.py benchmarks/ops/run.py tests/test_benchmark_distributions.py
  • All hooks passed.
• python scripts/check_header.py --check
  • All headers passed.
• GPU runner smoke:
  • FLA_BENCH_OP_WARMUP_ITERS=1 FLA_BENCH_WARMUP_MS=5 FLA_BENCH_REP_MS=10 python -m benchmarks.ops.run --op chunk_gla chunk_gdn chunk_kda chunk_comba --input-profile realistic --seed 42 --modes fwd --custom-shapes '{"smoke":{"B":1,"T":256,"H":2,"D":64}}' --base main
  • All four ops completed successfully.
• Packed-length sample: 8,192 exact tokens across 32 sequences with min=49, median=211, p90=431.6, and max=1024.

Benchmark / NCU (kernel changes only)

Not applicable. This PR does not change kernel code or claim a kernel speedup; the RTX 5080 run above is a functionality smoke for the benchmark harness.

Breaking changes

None. New profiles are opt-in and existing defaults are unchanged.

Checklist

• I have read CONTRIBUTING.md and follow its conventions (code style, docstrings, commit prefixes).
• I have read AGENTS.md and, where my change matches its scope, the relevant skill under .agents/skills.
• This is not a minor/cosmetic-only PR (typo, formatting, style-only tweaks).
• Dependent tests pass locally or in CI; new behavior is covered by tests where applicable.
• Kernel changes include same-hardware before/after benchmark numbers where applicable. This item is not applicable here because no kernel code changed.